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Abstract — We  investigate  the  gaps  for  Soldiers  in  information 
collection  and  resource  management  for  Intelligence, 
Surveillance,  and  Reconnaissance  (ISR).  ISR  comprises  the 
intelligence  functions  supporting  military  operations;  we 
concentrate  on  ISR  for  physical  sensors  (air  and  ground 
platforms).  To  identify  gaps,  we  use  approaches  from  Human 
Factors  (interactions  between  humans  and  technical  systems  to 
optimize  human  and  system  performance)  at  the  level  of  Soldier 
functions/activities  in  ISR.  Key  gaps  (e.g.,  the  loud  auditory 
signatures  of  some  air  assets,  unofficial  ISR  requests,  and 
unintended  battlefield  effects)  are  identified.  These  gaps  illustrate 
that  ISR  is  not  purely  a  technical  problem.  Instead,  interactions 
between  technical  systems,  humans,  and  the  environment  result 
in  unpredictability  and  adaptability  in  using  technical  systems. 
To  mitigate  these  gaps,  we  provide  technology  recommendations. 

Keywords — intelligence,  surveillance,  and  reconnaissance; 
ISR;  human  factors;  human-systems  integration;  cognitive  systems 
engineering;  intelligence 

I.  Introduction 

Intelligence,  surveillance,  and  reconnaissance  (ISR)  is  the 
“...‘hub’  of  21sl  Century  (Military)  Operations”  [1],  ISR 
comprises  the  integrated  intelligence  functions  supporting 
military  operations  [2].  The  U.S.  Army  conceptualizes 
functions  of  the  intelligence  cycle  as  information:  Collection, 
Processing,  Exploitation,  and  Dissemination  (CPED)  [3];  the 
U.K.  military  also  conceptualizes  an  intelligence  cycle  similar 
to  CPED.  There  are  two  distinct  sources  for  information 
collection  [4]: 

1.  Soft  sources :  Information  from  humans  (e.g.,  human 
terrain  mapping,  an  interview  with  a  confidential 
informant,  and  social  media). 

2.  Hard  sources'.  Information  from  physical  sensors 
(e.g.,  visible  imagery  captured  by  an  unmanned  aerial 
vehicle). 

This  paper  primarily  focuses  on  hard  sources  of  information. 


This  research  was  sponsored  by  the  U.S.  Army  Research  Laboratory  and 
the  U.K.  Ministry  of  Defence  and  was  accomplished  under  Agreement 
Number  W91  INF-06-3-0001.  The  views  and  conclusions  contained  in  this 
document  are  those  of  the  authors  and  should  not  be  interpreted  as 
representing  the  official  policies,  either  expressed  or  implied,  of  the  U.S. 
Army  Research  Laboratory,  the  U.S.  Government,  the  U.K.  Ministry  of 
Defense,  or  the  U.K.  Government.  The  U.S.  and  U.K.  Governments  are 
authorized  to  reproduce  and  distribute  reprints  for  Government  purposes 
notwithstanding  any  copyright  notation  herein. 
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The  purpose  of  this  research  was  to  examine  information 
collection  and  resource  management  of  ISR  assets  (typically 
aerial  assets  but  also  ground  assets)  and  the  battlefield  effects 
of  ISR  based  on  Soldier  goals,  constraints,  and  priorities.  In 
addition,  we  provide  technology  recommendations  for  many 
of  the  identified  gaps.  Gaps  were  identified  based  on  semi- 
structured  interviews  with  subject  matter  experts  (U.S.  Army 
Soldiers  with  ISR  experience  during  deployment)  using 
approaches  from  Human  Factors.  Human  Factors’  is  defined 
as  understanding  the  human  interactions  with  systems  (e.g., 
technical  systems,  communication  with  other  humans)  to 
optimize  both  human  performance  and  overall  system 
performance  [7],  [8],  Human  Factors  is  an  integral  part  of  the 
successful  development  and  implementation  of  technology 
(i.e.,  supporting  and  enhancing  human  cognitive  performance) 
because  of  the  [8],  [9]: 

1 .  Fundamental  limits  of  human  performance. 

2.  Large  number  of  ways  humans  can  interact  or  use 
(technical  and  other)  systems  in  dynamic  operational 
environments,  neither  of  which  may  be  anticipated 
without  an  understanding  of  goals,  activities,  and 
tasks. 

ISR  is  not  just  a  technical  problem;  humans  must  make 
decisions  about  information  collection  and  resource 
management.  Thus,  ISR  involves  social,  natural,  and  technical 
systems.  Social  and  natural  systems  tend  to  be  less 
predictable,  but  more  adaptable,  than  engineered  (technical) 
systems  [10],  The  different  systems  in  ISR  are: 

1.  Social  and  natural  systems:  Individual  humans  and 
groups  (military,  civilians,  and  insurgents). 

2.  Natural  systems:  Environmental  characteristics,  such 
as  the  weather,  terrain,  and  time  of  day. 

3.  Technical  systems:  Sensors  and  sensor  platforms, 
communication  devices  and  networks,  and  software 


!We  use  the  general  definition  of  Human  Factors  to  encompass  the  wide 
variety  of  other  highly  related  disciplines  or  terms.  Popular  examples  include 
Cognitive  (Systems)  Engineering,  Human-Centered  Computing,  and  Human- 
Systems  Integration  [5].  Different  areas  do  have  nuanced  distinctions,  but,  in 
our  opinion,  they  can  be  conceptualized  as  different  levels  of  analysis  for  the 
same  overarching  question  or  problem  [6].  In  Europe,  “Ergonomics”  is 
commonly  used  in  place  of  “Human  Factors.” 
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systems  for  collecting  information  and  managing 
resources. 

Because  ISR  includes  more  than  just  technical  systems,  it  has 
a  large  problem  space.  Gaps  for  Soldiers  may  result  from 
multiple  systems  and  their  interactions.  Consequently,  not  all 
gaps  can  be  identified  from  a  technical  approach.  An 
understanding  of  Soldier  activities  and  goals  is  needed  to 
determine  the  gaps  for  Soldiers  in  the  ISR  work  domain. 

In  Section  II,  we  cover  past  ISR  work,  which  is  mainly 
technology  focused  or  an  assessment  from  a  military 
command  perspective.  In  Section  III,  we  discuss  the  study 
methodology,  subject  matter  experts,  and  the  procedure, 
followed  by  the  gaps.  In  Section  IV,  to  meet  the  identified 
gaps,  we  provide  recommendations  for  developing  and 
implementing  technology.  In  Section  V,  we  conclude  the 
paper  and  discuss  limitations  and  future  directions. 

II.  Past  Work 

Past  work  in  ISR  is  mainly  technology  focused  or  an 
assessment  from  a  military  command  perspective.  For  long¬ 
term  ISR  technology,  several  roadmaps  provide  research  and 
development  plans  for  better  sensors,  improved  sensor 
platforms,  and  new  physical  network  capabilities  [11-14]. 
General  near-term  technology  work  seeks  to  address  gaps  in 
the  management,  processing,  and  fusion  of  heterogeneous 
(i.e.,  soft  and  hard)  information  to  aid  human  decision-making 

[15] ,  Using  a  technical  system  optimization  approach, 
multiple  capability  gaps  have  been  conceptually  identified 

[16] ;  these  gaps  include  the  need  for  a  common  operating 
picture  for  assets,  system  interoperability,  a  system  to 
determine  asset  suitability  for  collection  requirements,  and 
decision  aids  [16]. 

Specific  technology  efforts  for  ISR  concentrate  on 
different  computing  and  network  architectures  (e.g., 
scalability,  security,  and  bandwidth)  to  exploit  the  vast  and 
growing  amounts  of  data  [16],  [17],  There  is  also  a  broad 
research  program  on  techniques  for  soft  and  hard  information 
fusion  [18].  The  U.S.  and  U.K.  International  Technology 
Alliance  in  Network  &  Information  Sciences  is  developing 
technology  for  coalition  ISR  allocation  using  controlled 
natural  language  [19],  such  as  the  Sensor  Assignments  to 
Mission  (SAM)  system  [20],  [21].  Last,  comprehensive 
evaluations  of  tactical  and  operational  gaps  in  ISR  provide  a 
military  command  and  doctrinal  perspective  [22],  [23], 

III.  Human  Factors:  ISR  Gaps 

In  this  section,  we  discuss  the  study  methodology,  subject 
matter  experts,  and  the  procedure,  followed  by  the  gaps.  To 
provide  structure  to  the  interview  questions,  we  apply 
elements  from  two  functional/goal-oriented  Human  Factors 
approaches  [24-26].  The  general  aim  was  to  understand,  “Why 
is  a  user  performing  an  activity,  task,  action,  or  operation  in 
the  first  place?”  [24,  pp.  15];  this  approach  is  Activity- 
Centered  Design  [25].  A  related  approach,  Work  Domain 
Analysis,  was  also  used  to  determine  Soldier  functions,  goals, 


constraints,  and  priorities  [26],  These  approaches  provided  the 
overarching  structure  for  interviews  (see  Table  II). 

A .  Subject  Matter  Experts 

Fourteen  U.S.  Army  Soldiers  with  ISR  experience  were 
interviewed.  All  Soldiers  had  deployed  experience  with  ISR 
ranging  from  management,  collection,  and  analysis  duties  in  a 
Tactical  Operations  Center  to  first-hand  tactical  experience 
with  tactical  use  of  ISR  and  the  direct  effects  of  ISR  on 
military  operations.  Subject  matter  experts  consisted  of  13 
males  and  1  female  (mean  age  =  27.1  years  old,  age  unknown 
for  4  participants).  Characteristics  of  subject  matter  experts 
are  described  in  Table  I. 

TABLE  I. 

Characteristics  of  Subject  Matter  Experts 


Rank" 

Military 

Occupational 

Description  of  Deployed 
Experience0 

COL 

(retired) 

Equivalent  to 

3 5 -series 

Intelligence  OIC  at  DIV, 
BDE,  and  BN 

CPT 

35D 

BN  Intelligence  OIC 

CPT 

35D 

BDE  Intelligence 
Collection  Manager 

CPT 

35D 

BN  Intelligence  OIC, 
Intelligence  Operations 
Analyst 

CPT 

35F 

BN  Intelligence  OIC 

1LT 

11A 

Intelligence  Advisor  to 
Host  Nation 

1LT 

11A 

Intelligence  Advisor  to 
Host  Nation 

1LT 

35D 

BN  Intelligence  OIC 

SSG 

35F 

DIV  Intelligence 
Operations  Analyst,  BDE 
Collection  Management 

SSG 

29E 

BN  Electronic  Warfare 

SGT 

13B 

Targeting,  BN  Blue  Force 
Tracker 

SGT 

35F 

BDE  ISR  Operations 
NCOIC 

SGT 

35F 

BN  Intelligence  Analyst 

SGT 

35F 

BN  Intelligence  OIC, 
Targeting,  Operations 
Analyst 

'  Rank  descriptions:  http://www.armv.mil/svmbols/armyranks.html 

k  Military  Occupational  Specialty  descriptions: 
www.apd.armv.mil/Home/Links/PDFFiles/MOSBook.pdf 

'  Any  potentially  identifying  information  has  been  omitted;  the  descriptions  of  operational  experience 
are  incomplete.  Acronyms  for  military  echelons  (unit  sizes):  DIV,  BDE,  BN,  and  CO,  respectively, 
stands  for  Division,  Brigade,  Battalion,  and  Company.  For  a  description  of  military  echelons,  see 
http://en.wikipedia.org/wiki/Military  unit#Commands.2C  formations.2C  and  units 

OIC  is  Officer  in  Charge  and  NCOIC  is  Non-Commissioned  Officer  in  Charge. 

The  Intelligence  OIC  is  also  colloquially  referred  to  as  the  “S2.” 

B.  Procedure 

Subject  matter  experts  were  recruited  at  an  Umbrella  Week 
(this  is  a  scheduled  week  where  units  set  aside  times  for 
researchers  to  interview  Soldiers  and  administer  surveys)  and 
by  asking  other  researchers  and  Soldiers  for  suggested 
contacts.  All  Soldiers  were  told  that  participation  was 
completely  voluntary,  they  could  withdraw  at  any  time  and  for 
any  reason,  and  responses  were  non-attributional.  Subject 
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matter  experts  received  no  compensation  for  their  TABLE  III. 
participation. 


ISR  Gaps 


The  first  author  conducted  all  interviews.  Subject  matter 
experts  were  told  the  purpose  of  the  interview  was  to  identify 
the  tactical  and  operational  gaps  in  ISR,  based  on  their 
expertise  and  knowledge,  with  the  end  goal  of  improving  the 
effectiveness  of  ISR.  In  addition,  Soldiers  were  informed  that 
their  blunt,  honest  feedback  would  be  appreciated.  Some 
Soldiers  (6  out  of  14)  completed  a  15-minute  ISR  decision¬ 
making  task  before  the  interview.  Results  from  this  task  are 
not  discussed  here.  Examples  of  the  standardized  interview 
questions,  for  general  Soldier  functions,  are  shown  in  Table  II. 

TABLE  II. 

Example  Interview  Questions 

1 )  Requesting  and  Managing  ISR 

•  What  were  the  biggest  challenges? 

•  Did  you  use  workarounds  to  request  and  manage  ISR  (such 
as  deviations  from  standard  operating  procedures)?  If  so,  how 
and  why? 

2)  For  sensor  capabilities,  what  are  the  things  the  NIIRS  scaled  does  not 

take  into  account  (i.e.,  its  limitations)?  For  example,  effects  on  the 
battlefield. _ 

3)  How  and  when  (examples)  does  ISR  tend  to  be  effective  in  the  tactical 
and  operational  environment?  Ineffective? 

4)  Systems  (mostly  software) 

•  What  systems  did  you  use  to  request,  plan,  and  manage  ISR 
and  view  collected  information? 

_ »  What  capabilities  would  you  like  to  see  in  future  systems? 

NIIRS  stands  for  National  Imagery  Interpretability  Rating  Scale.  It  is  an  empirically  validated  scale  for 
characterizing  the  quality  and  performance  of  imagery  sensors  based  on  human  analyst’s  accuracy  in 
target  detection  and  target  identification.  See:  http://www.fas.org/irp/imint/niirs.htm  Note,  NIIRS 
includes  multispectral  imagery  (visible,  radar,  and  infrared). 

The  example  questions  were  always  asked,  but  interviews 
were  not  limited  to  these  questions.  Follow-up  questions  were 
asked  to  obtain  more  detailed  information  and  clarification.  To 
verify  understanding,  every  effort  was  made  to  ask  open- 
ended  as  opposed  to  potentially  leading  or  loaded  questions. 
For  security  reasons,  interview  responses  containing 
potentially  sensitive  information  (e.g.,  tactics,  techniques,  and 
procedures  or  specific  system  capabilities)  are  either  described 
generically  or  have  been  intentionally  omitted  at  the  discretion 
of  the  first  author. 

To  save  time  and  promote  discussion  among  the  Soldiers, 
a  combination  of  interview  formats  were  used: 

1.  Small  group  interviews:  In  groups  of  two,  four,  and 
four  Soldiers. 

2.  Individual  interviews  with  four  Soldiers. 

Six  Soldiers  (four  individual  and  a  group  of  two)  were 
interviewed  over  the  phone,  the  remaining  were  interviewed  in 
person.  The  group  and  individual  interviews  lasted  from  20 
minutes  to  2.5  hours  (mean  duration  =  70.3  minutes). 

C.  Gaps 

The  interviews  produced  nine  general  gaps.  These  gaps  are 
presented  in  Table  III  (the  numbers  in  parentheses  denote  the 
correspondence  to  interview  questions  in  Table  II).  Following 
Table  III,  gaps  in  software  systems  are  discussed. 


Gap 

Description 

Reason(s) 

Impact(s) 

Barriers  to 

coalition  and 

echelon 

intelligence 

sharing 

(1,3,4) 

Classified 
information 
(especially  Top 
Secret)  cannot, 
typically,  be 
easily  shared 
with  coalition 
partners  or 
lower  echelons. 
There  is  an 
inexorable 
trade-off 
between 
maintaining 
security  and 
sharing 
intelligence. 

There  is  no 
automated  or 
semi-automated 
system  for 
reducing  the 
classification 
level  of 
intelligence 
(may  take  weeks 
or  months  to 
reduce  the 
classification 
level  or  for 
declassification). 
Maintain 
security  to 
protect  sensitive 
information. 

Information 
collected  from 
assets  with  highly 
classified 

capabilities  may  not 
be  directly  shared 
with  lower 
echelons. 

Limits  intelligence 
sharing  between 
coalition  partners 
and  echelons. 

Loud 
unmanned 
aerial  vehicles 
(Soldiers  use 
the  term 
“flying 

lawnmowers”) 

(2,  3) 

A  few 

unmanned  aerial 
vehicles  have  a 
distinct  buzzing 
sound  that  is 
often  audible 
from  the  ground. 

Unmanned  aerial 
vehicle  operators 
are  sometimes 
unaware  of  noise 
given  the 
altitude,  terrain, 
and  weather. 
Certain 

platforms  simply 
are  much  louder 
than  other  ones. 

Unmanned  aerial 
vehicles  are  audible 
to  individuals  on 
the  ground;  hence, 
detection  of  ISR 
can  be  undesirable 
(e.g.,  draws 
attention  to  Soldiers 
nearby)  and  also 
desirable  (the  air 
version  of  a  ground 
“presence  patrol”). 

No  Common 
Operating 

Picture  for  ISR 
(1,3,4) 

No  single 

Mission 

Command 

System  for  ISR 
(air  and  ground 
assets).  This 
refers  to  asset 
locations,  rather 
than  information 
fusion,  the  latter 
is  addressed 
below. 

Limited  system 
interoperability 
(air  and  ground 
assets  are  on 
separate 

systems).  Not  all 
air  assets  are  on 
the  same  system. 
Coalition  forces 
often  use 
different 
systems. 

Asset  allocation 
may  be  suboptimal 
because  of  limited 
awareness  of 
location  between 
echelons  and 
coalition  partners. 
Upper  echelons 
disproportionately 
rely  on  air  assets. 

Individual 
differences  in 
the  operator 
performance  of 
unmanned 
aerial  vehicles 
(1,2,  3) 

Some  expert 
operators  could 
keep  platforms 
in  the  air  longer 
and  collect 
higher-quality 
information 
(e.g.,  use 
sweeping 
circular  flight 
paths  instead  of 
staying  directly 
on  top  of  an 
area). 

Speculatively,  a 
combination  of 
differences  in 
cognitive  ability, 
training,  and 
operational 
experience. 

The  effectiveness 
of  ISR  on  the 
battlefield, 
including  how  long 
air  platforms  are 
available,  the 
quality  and 
relevance  of 
collected 
information,  and 
the  likelihood  of 
platform  detection 
by  the  red  force. 

Purpose  of 
information 
collection? 

(1,2,  3,  4) 

It  is  extremely 
rare  to  find 
useful 
intelligence 
without  prior 
information 
(i.e.,  formalized 
as  information 
requirements). 

Too  much 
information 
from  disparate 
sources 
(information 
fusion  problem) 
and  lack  of 
system 

interoperability. 

Without 

Without  a  purpose 
information 
collection  is 
unlikely  to  be 
pertinent  and  ISR  is 
not  likely  to  be 
effective. 
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Gap 

Description 

Reason(s) 

lmpact(s) 

collection 
requirements 
finding 
something 
important  by 
chance  is  like 
finding  a 
“needle  in  a 
haystack.” 

Signals 

Intelligence 

(SIGINT) 

(1,2,  3,  4) 

SIGINT  (cell 
phones  and 
radio)  intercepts 
can  be 
incredibly 
informative  (i.e., 
actionable 
intelligence), 
but  not  enough 
platforms  have 
this  capability. 

SIGINT  red 
forces  frequently 
use  cell  phones 
to  communicate. 
SIGINT  is  a 
relatively  new 
capability. 

Increased  SIGINT 
is  likely  to  increase 
actionable 
intelligence  and 
help  fill  in  various 
intelligence  gaps. 

Soldier 
knowledge  of 
sensor  and 
platform 
capabilities 
(2,3) 

Mixed 

knowledge  of 
capabilities 
(NIIRS  scale 
and  other 
capabilities). 

Some  Soldiers 
self-reported 
that  they  had 
expert 
knowledge; 
others  indicated 
they  should 
know  the 
specific 
capabilities  of 
assets,  but  did 
not. 

Lack  of  formal 
training  (for 
many  non¬ 
intelligence 
Military 
Occupational 
Specialties). 

Gaps  between 
training  and 
conditions  on 
the  battlefield 
(acquiring  expert 
knowledge  may 
require 
substantial 
operational 
experience). 
Sensors  and 
platforms 
change  (e.g., 
updated  sensor 
packages). 

Asset  allocation 
may  be  suboptimal 
(over  and  under 
allocation  in  terms 
of  sensor  and 
platform 
capabilities). 

Specialized 
technical 
knowledge  and 
skills  needed  to 
configure 
ground  sensors 
(3,4) 

Few  Soldiers 
can  configure 
ground  sensors 
(typically  this 
requires  a  Field 
Service 

Representative). 

Requires  highly 

specialized 

technical 

knowledge 

(sensors  have 

disparate, 

proprietary 

programming 

interfaces). 

No  common 
interface  (e.g., 
no  general 
architecture)  for 
configuring 
ground  sensors. 

Ground  sensors 
may  have  reduced 
effectiveness  (e.g., 
not  configured  for 
key  alerts)  or  even 
go  unused  (e.g.,  too 
many  false 
positives). 

Unofficial  ISR 
requests 
(Soldiers  call 
these  ‘drug 
deals’) 

(1,3) 

Off  the  books 

ISR  requests 
(not  put  through 
the  official 
chain  of 
command). 

Sometimes 
requests  take  too 
long  to  get 
fulfilled  (e.g., 
must  be  put  in 
two  weeks  in 
advance)  or  go 
unfulfilled  with 
no  explanation 
using  the  official 
process. 

The  method  for 
putting  in 

Potentially  saves 
time  and  improves 
ISR  coverage. 

Can  greatly  speed 
up  the  time  it  takes 
to  get  ISR  (from  up 
to  two  weeks  down 
to  minutes). 

Although  rare,  this 
method  allows 
Soldiers  to  obtain 

ISR  assets  owned 
by  echelons 

Gap 

Description 

Reason(s) 

Impact(s) 

requests  is  not 

necessarily 

standardized. 

multiple  levels 
above  them. 

ISR  Software  Systems 

The  general  issue  with  ISR  software  systems  was  a  lack  of 
integration.  Subject  matter  experts  noted  using  two  main 
software  systems  to  manage  ISR:  Google  Earth2  and  a  North 
Atlantic  Treaty  Organization  (NATO)  system.  Google  Earth 
was  used  to  perform  the  following  functions: 

1.  Flight  planning:  Plan  and  share  planned  air  ISR  tracks. 

2.  Common  Operating  Picture:  Real-time  air  ISR 
tracking,  limited  to  specific  air  platforms  and  ones 
equipped  with  functioning  trackers. 

3.  Sensor  Feeds:  View  real-time  feeds  (when  supported). 
The  NATO  system  was  Interim  Geo-Spatial  Intelligence  Tool 
(IGeoSIT)  [27].  IGeoSIT  was  used  in  similar  ways  to  Google 
Earth  and  had  many  of  the  same  limitations.  In  addition  to 
software,  white  boards,  magnetic  boards,  and  paper  were 
commonly  used  to  manage  air  asset  patrol  schedules  (i.e.,  the 
ISR  synchronization  matrix)  and  to  keep  track  of  the  location 
of  air  assets. 

IV.  Recommendations 

To  mitigate  the  identified  gaps,  we  provide 
recommendations  for  developing  and  implementing 
technology.  Specific  recommendations  are  provided  for  some 
of  the  gaps  highlighted  above.  In  unpredictable,  dynamic 
work  domains  (such  as  ISR),  we  contend  that  enhancing 
human  performance  requires  technical  systems  that  are 
adaptive,  interactive,  integrated  (as  few  unique  systems  as 
possible),  and  transparent  (see  [10],  [28]).  Decision  aids  may 
enhance  Soldier  decision-making  for  ISR  allocation  and 
resource  management,  but  new  technical  capabilities  need  to 
also  be  flexible  (e.g.,  ad-hoc  and  unofficial  ISR  requests). 
Although  we  were  unable  to  provide  recommendations  for  all 
of  the  gaps,  we  hope  our  list  will  be  a  helpful  guide  for  others. 

A.  Common  Operating  Picture  for  Air  and  Ground  Assets 

Existing  systems  do  not  provide  a  comprehensive  Common 
Operating  Picture.  Unit  location  is  not  necessarily  predictable 
and  for  ISR,  is  an  important  part  of  management  and 
allocation  (e.g.,  distance  to  the  target  or  area  of  interest).  A 
dynamic,  integrated  Common  Operating  Picture  would 
increase  Soldier  awareness  of  available  ISR  resources  and 
help  keep  “options  available”  as  opposed  to  committing  to  an 
irreversible  course  of  action.  The  issue  of  limited  information 
on  locations  of  ISR  units  between  echelons  and  coalition 
partners  is  due  to  different  systems,  lack  of  interoperability, 
security,  and  possibly  additional  issues.  Locations  of  ISR 
assets  are  often  reported  over  the  radio.  One  Soldier  had  an 
intriguing  suggestion:  using  technology  to  process  radio 
reports  of  unit  locations  to  update  asset  location  on  digital 
maps. 


2 

httpr/Avww.google.com/earth/index.html 
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B.  Adapative  Decision  Aids 

In  the  military  domain,  empirical  research  on  cognitive 
performance  with  decision  aids  is  rare.  A  notable  exception  is 
evidence  for  enhanced  cognitive  performance  in  Mission 
Command  with  specific  decision  aids  [29].  In  a  similarly 
unpredictable  and  safety-critical  domain,  health  care,  the 
effectiveness  of  decision  aids  for  clinician  performance  is 
mixed  [30],  The  mixed  results  in  health  care  are  partially 
attributed  to  mismatches  between  the  often  rigid 
implementation  decision  aids  with  unpredictable  work 
processes  [28],  Implementation  of  decision  aids  is  a  complex, 
multidimensional  problem  due  to  the  variety  of  decision  aids 
and  their  varying  levels  of  automation  [8]. 

Nevertheless,  decision  aids  may  help  mitigate  ISR  resource 
management  and  allocation  challenges.  In  particular,  it  would 
be  useful  to  provide  Soldiers  with  ISR  asset  recommendations 
based  on  rationale  for  matching  sensor  and  corresponding 
platform  capabilities  to  different  allocation  tasks.  Furthermore, 
decision  aids  may  be  expanded  to  assist  with  flight  planning  to 
help  novice  air  asset  operators  perform  like  experts  and  to 
represent  sensor  capabilities  (e.g.,  imagery,  SIGINT)  and  the 
auditory  signatures  of  air  assets  over  different  terrain,  weather, 
and  other  environmental  conditions. 

The  technological  approach  we  are  pursuing  for  decision- 
aiding  is  the  SAM  system  [20],  [21] — an  artificial  intelligence 
(AI)  system  for  ISR  resource  allocation.  SAM  uses  a 
controlled  natural  language  as  a  common  human  and  machine- 
readable  representation  of  knowledge,  thus  it  is  likely  to  have 
greater  transparency  to  humans  than  black  box  AI.  In  addition, 
an  interactive  conversational  interface  for  SAM  is  under 
development,  allowing  users  to  change  and  update  ISR 
allocation  tasks  [31].  We  plan  to  conduct  behavioral  research 
to  assess  and  iteratively  improve  SAM  for  human  cognitive 
performance  (see  [32]). 

C.  Security 

Partial  automation  of  security  may  increase  the  speed, 
quantity,  and  critically  the  quality  of  information  sharing.  One 
approach  that  we  are  investigating  is  using  controlled  natural 
language  to  automate  sharing  ISR  assets  between  coalition 
partners  and  echelons  [33];  this  approach  may  also  work  for 
automation  of  information  sharing  policies.  Soldiers  reported 
that  Palantir 3  (a  software  system  for  general  intelligence 
analysis)  has  security  and  data-sharing  policies,  automating 
sharing  of  information,  and  intelligence  products. 

D.  Other  Gaps 

Other  gaps  may  also  be  addressed  by  ongoing  technology 
efforts.  The  loud  auditory  signatures  of  air  platforms  may  be 
partially  mitigated  by  a  combination  of  technological 
advances  in  design  and  acoustic  modeling  [34].  For  example, 
this  could  be  accomplished  through  integration  of  acoustic 
modeling  to  depict  asset  auditory  signatures  in  flight  planning 
software  and  real-time  modeling  during  the  operating  of  air 


3  http ://www. palantir.com/solutions/intelligence/ 


platforms.  Signal  modeling  could  also  be  implemented  as 
decision  aids,  recommendations  to  improve  asset  “coverage” 
(e.g.,  visible  imagery,  SIGINT).  In  addition,  interoperability 
issues  with  configuration  and  alerts  from  ground  sensors  may 
be  minimized  with  open  architecture  standards  (e.g.,  Open 
Geospatial  Consortium’s  Sensor  Web  Enablement  Initiative4 
and  Terra  Harvest5).  Such  standards  may  enable  integration  of 
disparate  ISR  sensor  feeds  into  a  single  system. 

E.  Integration 

We  strongly  recommend  that  novel  technology  is  integrated 
into  an  existing  Mission  Command  System  instead  of  creating 
another  new  system.  Unless  a  new  Mission  Command  System 
is  truly  a  usable  “system  of  systems,”  the  introduction  of  yet 
another  system  is  likely  to  create  more  problems  for  Soldiers 
than  will  be  solved.  The  sheer  number  of  different  Mission 
Command  Systems  is  a  general  and  growing  problem  for  the 
military.  In  response  to  the  question  of  developing  new 
capabilities,  nearly  all  Soldiers  called  for  integration  with 
existing  systems.  Poor  system  interoperability  often  means  the 
same  information  must  be  manually  re-entered  or  “fat¬ 
fingered”  into  multiple  systems.  Stove-piping  of  systems  is 
likely  to  result  in  data-entry  errors  and  time  delays  in 
analyzing  and  disseminating  intelligence.  In  a  coalition 
context,  the  number  of  different  systems  is  likely  even  greater, 
further  exacerbating  this  problem. 

V.  Conclusion 

We  applied  approaches  from  Human  Factors  to  identify 
gaps  in  ISR  and  provided  recommendations  for  technology. 
The  wide  range  of  the  gaps  indicates  the  scope  of  the 
challenges  for  implementing  technology. 

This  paper  has  strengths  and  weaknesses.  The  strengths  are 
identifying  gaps  from  the  Soldier  perspective,  rather  than  only 
a  technological  one,  and  a  relatively  large  number  of  subject 
matter  experts  compared  to  similar  research.  A  weakness  is 
the  use  of  self-report,  qualitative  data  from  interviews. 
Stronger  empirical  inferences  can  be  made  from  other  research 
methods,  such  as  observational  data  and  objective, 
quantitative  data  (e.g.,  [35]).  Another  weakness  is  the  focus  on 
breadth  over  depth.  This  is  a  general  limitation  of  analysis  at 
the  more  abstract,  functional  or  activity,  level  compared  to  the 
specific  lower  level  of  tasks  and  cognitive  task  performance. 
Given  the  unpredictability  and  adaptability  of  Soldier 
activities  in  ISR,  combined  with  its  high  dimensionality 
(system  interactions),  we  assert  that  a  broad  understanding  of 
problems  is  an  informative  starting  point.  Our  work  has  high 
breadth,  identifying  general  gaps  in  ISR,  but  limited  detail  for 
gaps. 

Technology  has  enormous  potential  to  enhance  the 
effectiveness  in  ISR,  but  for  technology  to  be  effective  it  must 
provide  solutions  to  actual  Soldier  needs.  In  future  work,  we 
plan  to  research  human  decision-making  for  ISR  allocation 


4  http://www.opengeosDatial.orH/domain/swe 

5  http://www.terraliarvest.net/ 
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with  and  without  decision-aids  using  SAM.  Results  from 
future  work  may  help  iteratively  refine  the  capabilities  of 
technical  systems,  enhancing  human  performance. 
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